1. Field of the Invention
The present invention relates generally to interactive viewing systems. In particular, the present invention relates to tracking an arbitrary object in an augmented reality viewing system.
2. Description of the Related Art
Interactive viewing systems have found application in manufacturing automation and maintenance, surgical procedures, educational instruction, mechanical, architectural, and interior designs, multimedia presentations, and motion picture production. Such interactive viewing systems may display computer-generated overlay images, such as a rendering of annotations, blueprints, component parts, buildings, backgrounds, and other images, in a user's field-of-view of a real-world environment to provide information about the real-world objects.
One type of interactive viewing system is referred to as an augmented reality (AR) system. In an AR system, computer-generated images, or virtual images, may be embedded in or merged with the user's view of the real-world environment to enhance the user's interaction with, or perception of, the environment. The user's view of the environment may be captured in a video image of the environment, and displayed to the viewer with embedded computer-generated images. Similarly, the computer-generated images may be projected on a see-through screen positioned in the user's field of view of the environment or displayed as an overlay in the user's actual field of view of the environment. The computer-generated images provide a visual enhancement, such as two-dimensional overlays and three-dimensional models, of real-world objects. The computer-generated images may include a text annotation describing an object, instructions for operation of a device, a schematic of an item, or a map linked to objects in the user's field of view.
In a conventional AR system, a user views the environment through a head mounted display (HMD), which may include a camera for capturing video images of the environment, and a display for projecting those video images with computer-generated virtual images. The HMD may also be configured as a see-through device, which allows the user to view the environment and on which computer-generated images may be projected. For example, a virtual schematic may be displayed on a see-through screen in the HMD, so that the user may view the schematic as part of the user's view of the environment as the user moves, thereby augmenting the user's perception of the environment with the schematic.
For an augmentation to be effective, the virtual image needs to track its place in the environment as the user's field of view changes. A computer-generated virtual image should appear in the video image or on the see-through display as though it is properly located within the real-world environment. For example, an AR system may draw an electrical schematic over a circuit board in the user's view and maintain the correct position of the schematic with respect to the circuit board as the user's view changes. If the virtual image is attached to a moving real object, that real object also is tracked so that the virtual image may track the movement. The tracking of computer-generated images should also be dynamic such that changes in the environment do not affect the ability to track computer-generated virtual images.
Conventional AR systems use remote sensing applications such as optical, video, ultrasonic, or magnetic sensing equipment, in which the user's movement is recorded and/or calculated with respect to the environment. For example, a video image of the user's environment is captured to identify real-world objects using known object recognition algorithms, or pre-programmed points to retrieve information based on the identified objects. A combined display of a virtual image of the physical scene and information related to the identified objects may then be created. A tracking system may detect the user's eye movement and/or head movement to determine the user's field of view of the environment so that a computer-generated virtual image may be tracked with a selected object, such as a wall, floor or circuit board, and its location is tracked by the computer as the user's field of view changes. By tracking the movement of the user, the external equipment can compare the user's movement to a known or predetermined location of items within the field of view to determine the user's field of view from various points in the environment. By identifying the location of real-world objects, computer-generated virtual images may then be tracked with the selected object in the display of the environment. Such systems, however, may require a large amount of computer processing power and may require a costly tracking system worn by the user. Moreover, such systems also require knowledge or remote sensing of various objects in the environment in order to provide accurate tracking of a virtual image to the objects. Such AR systems are costly and do not allow a user to track a virtual image to an arbitrary object.
Accordingly, there is a need in the art for an improved AR system that provides tracking of a virtual image to arbitrary objects in the real-world environment, while still providing adequate accuracy, freedom of movement of the user, and simplicity of use.